The Dreaming Variational Autoencoder for Reinforcement Learning Environments

Reinforcement learning has shown great potential in generalizing over raw sensory data using only a single neural network for value optimization. There are several challenges in the current state-of-the-art reinforcement learning algorithms that prevent them from converging towards the global optima. It is likely that the solution to these problems lies in short- and long-term planning, exploration and memory management for reinforcement learning algorithms. Games are often used to benchmark reinforcement learning algorithms as they provide a flexible, reproducible, and easy to control environment. Regardless, few games feature a state-space where results in exploration, memory, and planning are easily perceived. This paper presents The Dreaming Variational Autoencoder (DVAE), a neural network based generative modeling architecture for exploration in environments with sparse feedback. We further present Deep Maze, a novel and flexible maze engine that challenges DVAE in partial and fully-observable state-spaces, long-horizon tasks, and deterministic and stochastic problems. We show initial findings and encourage further work in reinforcement learning driven by generative exploration.Comment: Best Student Paper Award, Proceedings of the 38th SGAI International Conference on Artificial Intelligence, Cambridge, UK, 2018, Artificial Intelligence XXXV, 201

Approximation of corner polyhedra with families of intersection cuts

We study the problem of approximating the corner polyhedron using intersection cuts derived from families of lattice-free sets in $\mathbb{R}^n$. In particular, we look at the problem of characterizing families that approximate the corner polyhedron up to a constant factor, which depends only on $n$ and not the data or dimension of the corner polyhedron. The literature already contains several results in this direction. In this paper, we use the maximum number of facets of lattice-free sets in a family as a measure of its complexity and precisely characterize the level of complexity of a family required for constant factor approximations. As one of the main results, we show that, for each natural number $n$, a corner polyhedron with $n$ basic integer variables and an arbitrary number of continuous non-basic variables is approximated up to a constant factor by intersection cuts from lattice-free sets with at most $i$ facets if $i> 2^{n-1}$ and that no such approximation is possible if $i \leq 2^{n-1}$. When the approximation factor is allowed to depend on the denominator of the fractional vertex of the linear relaxation of the corner polyhedron, we show that the threshold is $i > n$ versus $i \leq n$. The tools introduced for proving such results are of independent interest for studying intersection cuts

Three-loop HTL gluon thermodynamics at intermediate coupling

We calculate the thermodynamic functions of pure-glue QCD to three-loop order using the hard-thermal-loop perturbation theory (HTLpt) reorganization of finite temperature quantum field theory. We show that at three-loop order hard-thermal-loop perturbation theory is compatible with lattice results for the pressure, energy density, and entropy down to temperatures $T\simeq3\;T_c$. Our results suggest that HTLpt provides a systematic framework that can used to calculate static and dynamic quantities for temperatures relevant at LHC.Comment: 24 pages, 13 figs. 2nd version: improved discussion and fixing typos. Published in JHE

Frame dragging with optical vortices

General Relativistic calculations in the linear regime have been made for electromagnetic beams of radiation known as optical vortices. These exotic beams of light carry a physical quantity known as optical orbital angular momentum (OAM). It is found that when a massive spinning neutral particle is placed along the optical axis, a phenomenon known as inertial frame dragging occurs. Our results are compared with those found previously for a ring laser and an order of magnitude estimate of the laser intensity needed for a precession frequency of 1 Hz is given for these "steady" beams of light.Comment: 13 pages, 2 figure

Chiral perturbation theory in a magnetic background - finite-temperature effects

We consider chiral perturbation theory for SU(2) at finite temperature $T$ in a constant magnetic background $B$. We compute the thermal mass of the pions and the pion decay constant to leading order in chiral perturbation theory in the presence of the magnetic field. The magnetic field gives rise to a splitting between $M_{\pi^0}$ and $M_{\pi^{\pm}}$ as well as between $F_{\pi^0}$ and $F_{\pi^{\pm}}$. We also calculate the free energy and the quark condensate to next-to-leading order in chiral perturbation theory. Both the pion decay constants and the quark condensate are decreasing slower as a function of temperature as compared to the case with vanishing magnetic field. The latter result suggests that the critical temperature $T_c$ for the chiral transition is larger in the presence of a constant magnetic field. The increase of $T_c$ as a function of $B$ is in agreement with most model calculations but in disagreement with recent lattice calculations.Comment: 24 pages and 9 fig

The Dreaming Variational Autoencoder for Reinforcement Learning Environments

Reinforcement learning has shown great potential in generalizing over raw sensory data using only a single neural network for value optimization. There are several challenges in the current state-of-the-art reinforcement learning algorithms that prevent them from converging towards the global optima. It is likely that the solution to these problems lies in short- and long-term planning, exploration and memory management for reinforcement learning algorithms. Games are often used to benchmark reinforcement learning algorithms as they provide a flexible, reproducible, and easy to control environment. Regardless, few games feature a state-space where results in exploration, memory, and planning are easily perceived. This paper presents The Dreaming Variational Autoencoder (DVAE), a neural network based generative modeling architecture for exploration in environments with sparse feedback. We further present Deep Maze, a novel and flexible maze engine that challenges DVAE in partial and fully-observable state-spaces, long-horizon tasks, and deterministic and stochastic problems. We show initial findings and encourage further work in reinforcement learning driven by generative exploration.The Dreaming Variational Autoencoder for Reinforcement Learning EnvironmentsacceptedVersionNivå

A Minimal Threshold of c-di-GMP Is Essential for Fruiting Body Formation and Sporulation in Myxococcus xanthus

Generally, the second messenger bis-(3’-5’)-cyclic dimeric GMP (c-di-GMP) regulates the switch between motile and sessile lifestyles in bacteria. Here, we show that c-di-GMP is an essential regulator of multicellular development in the social bacterium Myxococcus xanthus. In response to starvation, M. xanthus initiates a developmental program that culminates in formation of spore-filled fruiting bodies. We show that c-di-GMP accumulates at elevated levels during development and that this increase is essential for completion of development whereas excess c-di-GMP does not interfere with development. MXAN3735 (renamed DmxB) is identified as a diguanylate cyclase that only functions during development and is responsible for this increased c-di-GMP accumulation. DmxB synthesis is induced in response to starvation, thereby restricting DmxB activity to development. DmxB is essential for development and functions downstream of the Dif chemosensory system to stimulate exopolysaccharide accumulation by inducing transcription of a subset of the genes encoding proteins involved in exopolysaccharide synthesis. The developmental defects in the dmxB mutant are non-cell autonomous and rescued by co-development with a strain proficient in exopolysaccharide synthesis, suggesting reduced exopolysaccharide accumulation as the causative defect in this mutant. The NtrC-like transcriptional regulator EpsI/Nla24, which is required for exopolysaccharide accumulation, is identified as a c-diGMP receptor, and thus a putative target for DmxB generated c-di-GMP. Because DmxB can be—at least partially—functionally replaced by a heterologous diguanylate cyclase, these results altogether suggest a model in which a minimum threshold level of c-di-GMP is essential for the successful completion of multicellular development in M. xanthus

The Prevalence of Latent Mycobacterium Tuberculosis Infection Based on an Interferon-γ Release Assay: A Cross-Sectional Survey Among Urban Adults in Mwanza, Tanzania.

One third of the world's population is estimated to be latently infected with Mycobacterium tuberculosis (LTBI). Surveys of LTBI are rarely performed in resource poor TB high endemic countries like Tanzania although low-income countries harbor the largest burden of the worlds LTBI. The primary objective was to estimate the prevalence of LTBI in household contacts of pulmonary TB cases and a group of apparently healthy neighborhood controls in an urban setting of such a country. Secondly we assessed potential impact of LTBI on inflammation by quantitating circulating levels of an acute phase reactant: alpha-1-acid glycoprotein (AGP) in neighborhood controls. The study was nested within the framework of two nutrition studies among TB patients in Mwanza, Tanzania. Household contacts- and neighborhood controls were invited to participate. The study involved a questionnaire, BMI determination and blood samples to measure AGP, HIV testing and a Quantiferon Gold In tube (QFN-IT) test to detect signs of LTBI. 245 household contacts and 192 neighborhood controls had available QFN-IT data. Among household contacts, the proportion of QFT-IT positive was 59% compared to 41% in the neighborhood controls (p = 0.001). In a linear regression model adjusted for sex, age, CD4 and HIV, a QFT-IT positive test was associated with a 10% higher level of alpha-1-acid glycoprotein(AGP) (10(B) 1.10, 95% CI 1.01; 1.20, p = 0.03), compared to individuals with a QFT-IT negative test. LTBI is highly prevalent among apparently healthy urban Tanzanians even without known exposure to TB in the household. LTBI was found to be associated with elevated levels of AGP. The implications of this observation merit further studies